Unsupervised Machine Learning Approach

Methodology: DBSCAN

Univariate Temperature Data


We will grab a particular window in 2019 of just over two months

We still have a fair amount of data (given sensors record every 25s)

- - - Create small window for speed, plotting numerous kpi values - - -



Capturing THE polar vortex

image.png

Help on class DBSCAN in module sklearn.cluster._dbscan:

class DBSCAN(sklearn.base.ClusterMixin, sklearn.base.BaseEstimator)

   DBSCAN(eps=0.5, *, min_samples=5, metric='euclidean', metric_params=None, algorithm='auto', leaf_size=30, p=None, n_jobs=None)

   Perform DBSCAN clustering from vector array or distance matrix.

   DBSCAN - Density-Based Spatial Clustering of Applications with Noise.
   Finds core samples of high density and expands clusters from them.
   Good for data which contains clusters of similar density.

   Read more in the :ref:`User Guide <dbscan>`.

   Parameters
   ----------
   eps : float, default=0.5
       The maximum distance between two samples for one to be considered
       as in the neighborhood of the other. This is not a maximum bound
       on the distances of points within a cluster. This is the most
       important DBSCAN parameter to choose appropriately for your data set
       and distance function.

   min_samples : int, default=5
       The number of samples (or total weight) in a neighborhood for a point
       to be considered as a core point. This includes the point itself.

   metric : string, or callable, default='euclidean'
       The metric to use when calculating distance between instances in a
       feature array. If metric is a string or callable, it must be one of
       the options allowed by :func:`sklearn.metrics.pairwise_distances` for
       its metric parameter.
       If metric is "precomputed", X is assumed to be a distance matrix and
       must be square. X may be a :term:`Glossary <sparse graph>`, in which
       case only "nonzero" elements may be considered neighbors for DBSCAN.

       .. versionadded:: 0.17
          metric *precomputed* to accept precomputed sparse matrix.

   metric_params : dict, default=None
       Additional keyword arguments for the metric function.

       .. versionadded:: 0.19

   algorithm : {'auto', 'ball_tree', 'kd_tree', 'brute'}, default='auto'
       The algorithm to be used by the NearestNeighbors module
       to compute pointwise distances and find nearest neighbors.
       See NearestNeighbors module documentation for details.

   leaf_size : int, default=30
       Leaf size passed to BallTree or cKDTree. This can affect the speed
       of the construction and query, as well as the memory required
       to store the tree. The optimal value depends
       on the nature of the problem.

   p : float, default=None
       The power of the Minkowski metric to be used to calculate distance
       between points. If None, then ``p=2`` (equivalent to the Euclidean
       distance).

   n_jobs : int, default=None
       The number of parallel jobs to run.
       ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context.
       ``-1`` means using all processors. See :term:`Glossary <n_jobs>`
       for more details.


   Attributes
   ----------
   core_sample_indices_ : ndarray of shape (n_core_samples,)
       Indices of core samples.

   components_ : ndarray of shape (n_core_samples, n_features)
       Copy of each core sample found by training.

   labels_ : ndarray of shape (n_samples)
       Cluster labels for each point in the dataset given to fit().
       Noisy samples are given the label -1.

   Examples
   --------
   >>> from sklearn.cluster import DBSCAN
   >>> import numpy as np
   >>> X = np.array([[1, 2], [2, 2], [2, 3],
   ...               [8, 7], [8, 8], [25, 80]])
   >>> clustering = DBSCAN(eps=3, min_samples=2).fit(X)
   >>> clustering.labels_
   array([ 0,  0,  0,  1,  1, -1])
   >>> clustering
   DBSCAN(eps=3, min_samples=2)

   See Also
   --------
   OPTICS : A similar clustering at multiple values of eps. Our implementation
       is optimized for memory usage.

   Notes
   -----
   For an example, see :ref:`examples/cluster/plot_dbscan.py
   <sphx_glr_auto_examples_cluster_plot_dbscan.py>`.

   This implementation bulk-computes all neighborhood queries, which increases
   the memory complexity to O(n.d) where d is the average number of neighbors,
   while original DBSCAN had memory complexity O(n). It may attract a higher
   memory complexity when querying these nearest neighborhoods, depending
   on the ``algorithm``.

   One way to avoid the query complexity is to pre-compute sparse
   neighborhoods in chunks using
   :func:`NearestNeighbors.radius_neighbors_graph
   <sklearn.neighbors.NearestNeighbors.radius_neighbors_graph>` with
   ``mode='distance'``, then using ``metric='precomputed'`` here.

   Another way to reduce memory and computation time is to remove
   (near-)duplicate points and use ``sample_weight`` instead.

   :class:`cluster.OPTICS` provides a similar clustering with lower memory
   usage.

   References
   ----------
   Ester, M., H. P. Kriegel, J. Sander, and X. Xu, "A Density-Based
   Algorithm for Discovering Clusters in Large Spatial Databases with Noise".
   In: Proceedings of the 2nd International Conference on Knowledge Discovery
   and Data Mining, Portland, OR, AAAI Press, pp. 226-231. 1996

   Schubert, E., Sander, J., Ester, M., Kriegel, H. P., & Xu, X. (2017).
   DBSCAN revisited, revisited: why and how you should (still) use DBSCAN.
   ACM Transactions on Database Systems (TODS), 42(3), 19.

   Method resolution order:
       DBSCAN
       sklearn.base.ClusterMixin
       sklearn.base.BaseEstimator
       builtins.object

   Methods defined here:

   __init__(self, eps=0.5, *, min_samples=5, metric='euclidean', metric_params=None, algorithm='auto', leaf_size=30, p=None, n_jobs=None)
       Initialize self.  See help(type(self)) for accurate signature.

   fit(self, X, y=None, sample_weight=None)
       Perform DBSCAN clustering from features, or distance matrix.

       Parameters
       ----------
       X : {array-like, sparse matrix} of shape (n_samples, n_features), or             (n_samples, n_samples)
           Training instances to cluster, or distances between instances if
           ``metric='precomputed'``. If a sparse matrix is provided, it will
           be converted into a sparse ``csr_matrix``.

       sample_weight : array-like of shape (n_samples,), default=None
           Weight of each sample, such that a sample with a weight of at least
           ``min_samples`` is by itself a core sample; a sample with a
           negative weight may inhibit its eps-neighbor from being core.
           Note that weights are absolute, and default to 1.

       y : Ignored
           Not used, present here for API consistency by convention.

       Returns
       -------
       self

   fit_predict(self, X, y=None, sample_weight=None)
       Perform DBSCAN clustering from features or distance matrix,
       and return cluster labels.

       Parameters
       ----------
       X : {array-like, sparse matrix} of shape (n_samples, n_features), or             (n_samples, n_samples)
           Training instances to cluster, or distances between instances if
           ``metric='precomputed'``. If a sparse matrix is provided, it will
           be converted into a sparse ``csr_matrix``.

       sample_weight : array-like of shape (n_samples,), default=None
           Weight of each sample, such that a sample with a weight of at least
           ``min_samples`` is by itself a core sample; a sample with a
           negative weight may inhibit its eps-neighbor from being core.
           Note that weights are absolute, and default to 1.

       y : Ignored
           Not used, present here for API consistency by convention.

       Returns
       -------
       labels : ndarray of shape (n_samples,)
           Cluster labels. Noisy samples are given the label -1.

   ----------------------------------------------------------------------
   Data descriptors inherited from sklearn.base.ClusterMixin:

   __dict__
       dictionary for instance variables (if defined)

   __weakref__
       list of weak references to the object (if defined)

   ----------------------------------------------------------------------
   Methods inherited from sklearn.base.BaseEstimator:

   __getstate__(self)

   __repr__(self, N_CHAR_MAX=700)
       Return repr(self).

   __setstate__(self, state)


   get_params(self, deep=True)
       Get parameters for this estimator.

       Parameters
       ----------
       deep : bool, default=True
           If True, will return the parameters for this estimator and
           contained subobjects that are estimators.

       Returns
       -------
       params : dict
           Parameter names mapped to their values.
RESULTS:
========    

db = DBSCAN(eps=0.03, min_samples=6, metric='euclidean', n_jobs=-1)
 0    139811
 3      4950
 2        37
 1        26
-1        19
 4        10


db = DBSCAN(eps=0.04, min_samples=5, metric='euclidean', n_jobs=-1)
 0    144840
-1        13


db = DBSCAN(eps=0.04, min_samples=7, metric='euclidean', n_jobs=-1)
 0    139848
 1      4989
-1        16


db = DBSCAN(eps=0.04, min_samples=7, metric='euclidean', n_jobs=-1)
 0    139848
 1      4989
-1        16


db = DBSCAN(eps=0.1, min_samples=10, metric='euclidean', n_jobs=-1)
 0    198364
 4      4990
 1        66
-1        41
 2        10
 3        10




``

Thus

  1. Lowering the eps value causes less and less anomalies to be determined
  2. Increasing the eps value then...








Repeating:






We will StandardScaler our data now

ITERATE 0:

ITERATE 1:

https://en.wikipedia.org/wiki/January%E2%80%93February_2019_North_American_cold_wave

https://weatherspark.com/m/14091/2/Average-Weather-in-February-in-Chicago-Illinois-United-States

Five records were set for Chicago for the month of January 2019:

image.png

plotting:

RESULTS:

db = DBSCAN(eps=0.01, min_samples=30, metric='euclidean', n_jobs=-1).fit(X)
 0    139272
 3      4284
 4       664
 2       403
-1       126
 1       104


BELOW SHOWN IN PLOT:

db = DBSCAN(eps=0.01, min_samples=20, metric='euclidean', n_jobs=-1).fit(X)
 0    139842
 1      4988
-1        23
NOT BAD !

image.png

Calculating Metrics:

Using Tuning Approach:





Appendix: Do Not Delete

from sklearn.cluster import DBSCAN
clustering1 = DBSCAN(eps=0.09, min_samples=6).fit(np.array(ts_dataframe['Normalized Profit']).reshape(-1,1))

labels = clustering1.labels_

outlier_pos = np.where(labels == -1)[0]

x = []; y = [];
for pos in outlier_pos:
    x.append(np.array(ts_dataframe['Normalized Profit'])[pos])
    y.append(ts_dataframe['Normalized Profit'].index[pos])

plt.plot(ts_dataframe['Normalized Profit'].loc[ts_dataframe['Normalized Profit'].index], 'k-')
plt.plot(y,x,'r*', markersize=8)  
plt.legend(['Actual', 'Anomaly Detected'])
plt.xlabel('Time Period')
plt.xticks([0, 20, 40, 60, 80, 99],[ts_dataframe.index[0],ts_dataframe.index[20], ts_dataframe.index[40], ts_dataframe.index[60], ts_dataframe.index[80], ts_dataframe.index[99]] ,rotation=45)
plt.ylabel('Normalized Profit')

image.png

image.png

image-2.png

image-3.png

image-4.png

image-5.png

Examining other sensors:

image.png